High luminance projection displays and associated methods

ABSTRACT

Projection displays include a highlight projector and a main projector. Highlights projected by the highlight projector boost luminance in highlight areas of a base image projected by the main projector. Various highlight projectors are based on steerable beams, holographic projectors, and spatial light modulators. A Fourier transform component and a mask positioned on the Fourier plane thereof are used to attenuate or eliminate selected spatial frequencies, e.g., to increase peak luminance without raising the black level of the projected image.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a Continuation of U.S. patent application Ser. No.15/147,238 filed May 5, 2016 which is a Continuation of U.S. patentapplication Ser. No. 14/112,347 filed Oct. 17, 2013 which is a nationalstage application of International Patent Application No.PCT/U2012/032995 filed Apr. 11, 2012, which claims priority to U.S.Provisional Application No. 61/476,949 filed Apr. 19, 2011, which arehereby incorporated by reference in their entirety for all purposes.

TECHNICAL FIELD

The invention relates to projection displays. An example embodimentprovides a digital cinema display. Other embodiments provide displayssuch as televisions, computer displays, and special purpose displayssuch as advertising displays, virtual reality displays, game displays,and medical imaging displays.

BACKGROUND

There is an increasing interest in providing displays that are capableof reproducing realistic-looking images. One aspect of achievingrealistic images is providing high peak luminance and high dynamicrange. A typical natural scene includes areas that are very bright, forexample the sun in the sky and highlights of brightly illuminatedobjects, as well as areas that are dim, for example objects in theshadows. Achieving realistic images of general scenes is not possible ondisplays that are incapable of high peak luminance.

Current projection technology does not scale efficiently to highluminance. For example, in many common projector designs, a lightsource, such as a xenon lamp, illuminates one or more spatial lightmodulators. The spatial light modulators direct some light to the screenwhile absorbing or redirecting other light. Achieving high luminancerequires scaling up the power of the light source. The increasing powerconsumption of the light source becomes an obstacle to increasing thebrightness of the light source to levels sufficient to provide peakluminance at a level typical of natural scenes. Moreover, a powerfullight source can cause problems with overheating spatial lightmodulators and other components in the projector, among other issues.

As an example, a current digital cinema projector may have a lightsource that consumes 8 kilowatts of electrical power to illuminate alarge screen producing a peak luminance of 48 nits (48 cd/m²). In orderto achieve a peak luminance of 12,000 nits (a luminance commonlyencountered in everyday life), the power of the light source would needto be scaled to over 2 megawatts. This is clearly impractical in mostcases.

A further problem which prevents significant increases in the peakluminance of many conventional projection displays is that the contrastdoes not increase with peak luminance. In many such displays, increasingthe intensity of the light source to achieve an increased peak luminancealso raises the black level. Therefore, attempts to increase the peakluminance past a threshold will result in an unacceptably high blacklevel.

A further obstacle to providing displays having high enough luminance topresent realistic images is that the response of the human visual systemto light is roughly logarithmic. By contrast, power requirements scaleroughly linearly with luminance. Doubling the luminance of an image,assuming the same efficiency of the light source, requires doubling thepower. However, doubling the luminance does not result in an image whichwill be perceived by a viewer as being twice as bright. Doubling theapparent brightness requires approximately squaring the luminance.

The foregoing examples of the related art and limitations relatedthereto are intended to be illustrative and not exclusive. Otherlimitations of the related art will become apparent to those of skill inthe art upon a reading of the specification and a study of the drawings.

SUMMARY

This invention has a range of aspects. Embodiments of the inventionprovide projection displays, methods for operating projection displays,dual-modulation displays, media containing computer-readableinstructions which, when executed by a data processor, cause the dataprocessor to execute a method according to the invention, methods fordisplaying images, and methods for processing image data for display,among others.

One example aspect of the invention provides a display systemcomprising: a main projector arranged to project an image defined bybase image data onto a screen and a highlight projector arranged toproject a highlight image defined by highlight image data onto thescreen in registration with the base image. An image processor isconfigured to process image data to generate the highlight image data.

In some embodiments, the highlight projector comprises a scanning beamprojector. The scanning beam projector may provide, for example, laserbeams of a plurality of primary colours (for example, red, green, andblue beams). The beams may be scanned together or independently to causethe highlight areas to have desired apparent brightness and colours. Inother embodiments the scanning beam projector provides a scannable beamof white light.

In some embodiments the highlight projector comprises a 2D holographicprojector.

Another aspect provides a highlight projector system comprising an imageprocessor configured to process image data to yield a highlight image;and a light projector operable to project light according to thehighlight image in registration with a base image.

Another example aspect provides a display comprising a light source ofspatially-modulated light arranged to illuminate a spatial lightmodulator wherein the source of spatially modulated light comprises a 2Dholographic light source.

Another example aspect provides a method for displaying an image definedby image data. The method comprises concentrating light from a lightsource to yield light that has been spatially-modulated in a mannerbased on the image data; illuminating a spatial light modulator with thespatially-modulated light; and controlling the spatial light modulatorto display an image according to the image data. Concentrating the lightmay comprise generating a computer-generated 2D hologram, for example.In some embodiments, the light comprises coherent light andconcentrating the light comprises adjusting phases of the light in theFourier plane of an optical system.

Another example aspect provides a method for displaying an imageaccording to image data. The method comprises processing the image datato generate a base image and a highlight image comprising highlightpixels; operating a main projector to display the base image; andoperating the highlight projector to display the highlight imagesuperposed with the base image.

In addition to the exemplary aspects and embodiments described above,further aspects and embodiments will become apparent by reference to thedrawings and by study of the following detailed descriptions.

BRIEF DESCRIPTION OF DRAWINGS

The accompanying drawings illustrate non-limiting embodiments of theinvention.

FIG. 1 is a schematic view of a display system according to an exampleembodiment.

FIG. 1A is a schematic view of a display system according to anotherexample embodiment.

FIGS. 2A and 2B are exemplary histograms showing the number of pixels inan image as a function of the luminance of those pixels for,respectively, a bright image and a dark image.

FIG. 3 is a schematic illustration of an example apparatus that combinesa highlight projector with a main projector.

FIG. 4 is a block diagram illustrating image data processing componentsof a display system according to an example embodiment.

FIG. 5 is a schematic view illustrating a highlight projector comprisinga light-redirecting projector according to an example embodiment.Highlight projectors of the type illustrated in FIG. 5 may be configuredto project light onto a screen, a component of a main projector or aspatial light modulator, for example.

FIG. 6A is a schematic view illustrating an example holographicprojector configured to project a desired highlight image onto a spatiallight modulator which is controlled to either redirect, dump, or absorblight in image areas outside of the highlight areas.

FIG. 6B is a schematic view illustrating a holographic projectorconfigured to project a highlight image directly onto a spatial lightmodulator of a main projector according to an example embodiment.

FIG. 7 is a schematic view illustrating a highlight projector comprisinga light source arranged to illuminate a 2D spatial light modulatoraccording to an example embodiment.

FIG. 8 is a schematic view illustrating a highlight projector comprisinga light source that illuminates a spatial light modulator with a spatialfilter in a Fourier plane for removing leakage light according to anexample embodiment.

FIG. 9 shows a display according to another embodiment. Displays havingthe overall architecture illustrated schematically in FIG. 9 may be usedas a stand-alone displays (e.g. as a television, computer monitor,special purpose display or the like) or as part of a display system thatincludes a highlight projector.

DETAILED DESCRIPTION

Throughout the following description specific details are set forth inorder to provide a more thorough understanding to persons skilled in theart. However, well known elements may not have been shown or describedin detail to avoid unnecessarily obscuring the disclosure. Accordingly,the description and drawings are to be regarded in an illustrative,rather than a restrictive, sense.

Some embodiments of the invention provide projection displays includinga main projector and a highlight projector. The main projector may havea relatively low peak luminance and may be used to project a full image.The luminance of highlights in the image projected by the main projectoris lower than desired. The highlight projector may project concentratedlight to boost luminance at the location(s) of highlights, therebyboosting the luminance of highlights significantly.

FIG. 1 shows a projection system 10 according to a first exampleembodiment. Projection system 10 includes a main projector 12 having alens 14 that projects an image 16 onto a screen 18. Screen 18 may be afront-projection screen or a rear-projection screen. System 10 furtherincludes a separate highlight projector 20 having a lens 22 whichprojects an image 16A onto screen 18. Images 16 and 16A are superposedso that a viewer sees an image resulting from the combination of images16 and 16A.

Main projector 12 may comprise any suitable image projector. Forexample, main projector 12 may comprise a DLP-based projector, aprojector which uses one or more liquid crystal on silicon (LCOS)spatial light modulators, a projector comprising a transmissive liquidcrystal display (LCD) panel to modulate light, a cathode ray tube (CRT)projector or the like.

Highlight projector 20 is of a type which can deliver concentrated lightto at least some areas within the area of image 16, preferably withoutsignificantly raising the light level in other areas within image 16.For example, highlight projector 20 may comprise one or more scanningbeams which can be directed to add further illumination to only selectedhighlight areas of image 16.

Highlight projector 20 and main projector 12 are co-registered so thathighlight projector 20 can accurately deliver additional light to smallhighlight areas within the image 16 projected by main projector 12. Insome embodiments highlight projector 20 has a spatial resolution equalto or greater than that of main projector 12. In other embodiments,projector 20 can have a spatial resolution less than main projector 12.In other embodiments, projector 20 can have a spatial resolution lessthan main projector 12.

In some embodiments, highlight projector 20 and an image processor areprovided for use as an add-on to an existing main projector such as acommercially-available digital cinema projector. The image processor maybe configured to receive image data for projection and generate ahighlight image for display by the highlight projector. The imageprocessor may, in some embodiments, modify the image data to provide abase image for display by the existing main projector. The highlightprojector may be calibrated upon installation to produce highlightimages that are registered with images produced by the existing mainprojector.

Advantageously, in typical scenes, only a relatively very smallproportion of the pixels in an image need to be displayed with aluminance greater than the peak luminance of standard projector 12 forenhanced realism. It has been found that enhanced realism can beachieved by providing bright highlights very selectively. FIGS. 2A and2B are histograms showing the number of pixels in an image as a functionof the luminance of those pixels for, respectively, a bright image and adark image prepared by a human colorist for viewing on a display havinga high peak luminance. In each case the colorist adjusted the image forwhat the colorist considered to be optimum appearance.

Somewhat surprisingly, the average brightness of all pixels in thebright image is still relatively very low. Even in the bright imagehaving the histogram of FIG. 3 , it can be seen that only a relativelyvery small proportion of the pixels have high luminances (e.g.luminances over about 1,000 or 2,000 nits). The few very bright pixelsat high and very high luminances can result in the image having a muchmore realistic appearance without significantly affecting the lightadaptation of a viewer's eyes. This differs from real scenes from naturein which many or all pixels may have very high luminance. For example, areal scene on a glacier on a sunny day may be so bright that it isuncomfortable or even harmful to view for extended periods without darksunglasses. A colorist may prepare such a scene in a way which resultsin a relatively low average luminance while providing high luminance ina few key areas to provide a more realistic viewing experience.

Some embodiments exploit the fact that even very bright scenes may berendered at a surprisingly low average luminance while preserving arealistic viewing impression if small highlight areas are presented witha peak luminance that is much higher than the average luminance withwhich the image is presented to viewers. Some embodiments use ahighlight projector that is much too low in power to raise all pixels ofimage 16 to the level of the brightest highlights but is capable ofboosting illumination of the highlights to the desired level. In suchembodiments, light from the highlight projector is concentrated into thehighlights to provide desired brightness in the highlights.

There is a wide variety of ways to arrange a main projector andhighlight projector in combination. For example, systems providing amain projector and a highlight projector for selectively boostingluminance of highlight areas may be provided which are arranged to haveany combination of the following features:

-   -   The main projector and the highlight projector may use the same        overall technologies or different technologies.    -   The main projector and highlight projector may be provided in        the form of separate units or in the form of a combined unit        (e.g., integrated form factor). Where the main projector and        highlight projector are provided in the form of a combined unit        the main projector and highlight projector may share certain        optical components and/or certain optical paths. For example,        the main projector and highlight projector may share one or more        of a projection lens, relay optics, one or more spatial light        modulators or the like. Various examples of shared components        and optical paths are set out below.    -   A system may comprise one or more than one main projectors that        collectively project a base image. For example, system 10 may        comprise a plurality of main projectors 12 that collectively        illuminate screen 18 to provide image 16.    -   A system may comprise one or more than one highlight projectors        that can collectively project a highlight image to boost        illumination of highlight areas. For example, highlight        projector 20 may comprise a plurality of units that can be        controlled to collectively direct light onto highlight areas of        image 16.    -   A highlight projector may be monochrome (e.g. may project white        light) or polychromatic.    -   A highlight projector may optionally include filtration (such as        for example a spatial filter in a Fourier plane) to suppress        illumination outside of highlight areas.    -   A highlight projector may optionally comprise one or more        spatial light modulators. The spatial light modulator(s) may be        controlled to perform one or more of: direct light to illuminate        highlight areas, correcting errors in a projected highlight        image, suppress illumination outside of highlight areas, adjust        a highlight image to blend smoothly into a base image projected        by a main projector, and redirect light from outside of        highlight areas into highlight areas. In embodiments in which        the highlight projector includes one or more spatial light        modulators, the spatial light modulator(s) may comprise spatial        light modulator(s) shared by the main projector and/or may        comprise spatial light modulator(s) dedicated to the highlight        projector.    -   The main projector and highlight projector may be arranged for        front-projection or rear-projection. It is not mandatory that        highlight projector 20 and main projector 12 illuminate screen        18 from the same side. In embodiments where screen 18 is        translucent (e.g. where screen 18 comprises a rear-projection        type of screen) highlight projector 20 and main projector 12 may        illuminate screen 18 from opposite sides.

These different approaches and their permutations and combinations arenot limiting but are intended to provide examples of some embodimentswithin the ambit of the invention.

Advantageously, the combined image, as viewed by a viewer includes somehighlights in which the peak luminance significantly exceeds the peakluminance of main projector 12. For example, the main projector may havea peak luminance of 500 nits or less while the highlight areas may havea peak luminance of 2000 nits or more. Some main projectors intended foruse in dark viewing environments (e.g. movie theaters) may provide peakluminance of 15 to 50 nits or so, for example. Some such projectors aredesigned to image onto large-area screens. Some main projectors intendedfor use in bright viewing environments may provide peak luminance of 100to 300 nits or so, for example.

Since the highlight areas illuminated by highlight projector 20 maycomprise only a very small fraction (e.g., less than 10%, 5%, 1%, oreven less than 0.1%) of the area of image 16, highlight projector 20 maybe able to achieve the desired high luminance in highlight areas withoutrequiring an impractical power input.

FIG. 1A shows a projector system according to an example embodiment inwhich a highlight projector comprises a spot light source 20A whichproduces narrow beam 21 of light and a deflector 23 comprising scanningmirrors 23A and 23B. Mirrors 23A and 23B are pivotally mounted andoperated by actuators (not shown) such that light beam 21 can be guidedto form a small spot 25 at any desired location in image 16. Theintensity of light beam 21 and the locations at which spot 25 isdisplayed may be controlled by a controller to achieve increasedluminance in selected highlight areas. In some embodiments, brightnessof a highlight area is controlled at least in part by varying the amountof time that spot 25 is controlled to dwell on the highlight area. Insome embodiments, brightness of a highlight area is controlled at leastin part by controlling the intensity and/or duty cycle of beam 21 whilethe beam 21 is illuminating the highlight area.

Beam 21 may, for example, comprise a laser beam. In some embodiments thehighlight projector comprises three laser beams of different colors thatcan be combined to make white highlights. For example the highlightprojector may comprise red, green and blue laser beams. In suchembodiments the beams may be steered to illuminate highlight areas by asingle deflector assembly (e.g. a single set of mirrors 23A, 23B). Inalternative embodiments a separate deflection assembly is provided foreach of a plurality of beams 21.

Since highlights only appear typically in a small proportion of theoverall area of an image 16, the laser may increase the perceivedbrightness of highlight areas by dwelling longer in those areas. Thelaser need not illuminate any parts of image 16 outside of the highlightareas.

In embodiments in which the highlight projector comprises a steerablelight beam, a controller which steers the light beam may be configuredto control mirrors 23A and 23B (or an alternative beam-steeringmechanism such as a mechanism using digital light deflectors, gratinglight valves or the like) to cause spot 25 to follow a trajectory thatis dependent on the locations of highlight areas to be illuminated. Itis not necessary for the beam steering mechanism to scan in a raster orother pattern that covers all pixels of image 16. By steering spot 25 ina trajectory that takes spot 25 to highlight areas while avoiding atleast some pixels that are outside of highlight areas, controller 25 cancause spot 25 to dwell on the highlight areas for periods sufficient toachieve a desired luminance of the highlight areas.

Main projector 12 and highlight projector 20 may optionally beintegrated with one another such that the two projectors share somecommon optical paths. For example, the optical systems of highlightprojector 20 and main projector 12 may be arranged to share a commonprojection lens 14. One example of this is illustrated in FIG. 3 . FIG.3 is schematic in nature. Optical components that may be present in theoptical paths such as relay lenses, mirrors, filters or the like havebeen omitted for clarity.

In the embodiment illustrated in FIG. 3 , main projector 12 comprises alight source 30 which can emit light 31 to illuminate a spatial lightmodulator 32. Light source 30 may comprise a uniform light source or alight source that can be spatially modulated in accordance with imagedata (for example a base image). Light modulated by spatial lightmodulator 32 is directed by projection lens 14 onto screen 18 (not shownin FIG. 3 ) to provide image 16 (not shown in FIG. 3 ).

In this embodiment, the highlight projector comprises a high-intensitynarrow beam light source 34 that can be controlled to emit a narrow beamof light 35 that is steered by a X-Y deflector 36 and optical combiner37 to produce a brightly illuminated spot 38 on spatial light modulator32. By controlling the intensity and/or turning on or off light source34 while scanning with X-Y scanner 36, a plurality of differenthighlight areas may be illuminated on spatial light modulator 32 withlight from light source 34. This additional light, as modulated byspatial light modulator 32, is also imaged by lens 14 to add to theluminance of highlight areas within image 16.

In an alternative embodiment, optical combiner 37 is located betweenspatial light modulator 32 and projection lens 14 such that spot 38 isprojected directly onto a screen 18. In this alternative embodiment theoptical paths of the main and highlight projectors may have onlyprojection lens 14 in common.

FIG. 4 is a block diagram illustrating image data processing componentsof a display system according to an example embodiment. An imageprocessing system 40 receives image data 42 and processes image data 42to identify highlight areas. Processing may comprise, for example,comparing pixel luminance values to a first threshold and identifying asbelonging to highlight areas those pixels having luminance values inexcess of the first threshold. In some embodiments highlight areas maybe limited to areas comprising a predetermined area of connected pixelshaving luminance values exceeding the first threshold. As anotherexample, processing may identify highlight areas as being made up of theM highest-luminance pixels (where M is a number) or those pixels thatare at or above the Nth percentile for luminance (where N is apercentile such as the 90^(th) percentile or 95^(th) percentile or98^(th) percentile or 99^(th) percentile or 99.9^(th) percentile).Processing may comprise applying a plurality of such criteria (forexample, highlight areas may be identified as the up-to M pixels forwhich luminance exceeds a threshold).

In some embodiments, processing comprises trading off peak luminanceagainst area included in highlight areas. Such processing may comprisehistogram analysis. For example, for an image in which processingidentifies a relatively large number of pixels as belonging to highlightareas according to a first criterion, the processing may select betweenretaining the highlight areas according to the first criterion at thecost of reduced peak luminance achievable in the highlight areas orapplying a second criterion to reduce the number of pixels included inhighlight areas. Such processing may comprise histogram analysis.

In some embodiments, processing is performed with reference to anadaptation point. The adaptation point may, for example, comprise or bedetermined from a logarithmic mean luminance of the image. In the caseof video images the adaptation-point may comprise a temporal averageover some previous images. In such embodiments, processing to identifyhighlight areas may comprise identifying pixels that have luminancehigher than the adaptation point by at least a threshold amount.

Image processing system 40 generates a highlight image 43 which isdelivered to highlight projector 20. Highlight image 43 is displayed byhighlight projector 20 to provide increased brightness in highlightareas. Pixels outside of the highlight areas may have very small or zerovalues in highlight image 43. Image processing system 40 also delivers abase image 44 for projection by main projector 12.

In some embodiments, base image 44 is the same as image data 42. Inother embodiments, base image 44 is processed to provide a smoothtransition between highlighted areas which are illuminated primarily byhighlight projector 20 and base areas of image 16 which are illuminatedprimarily or entirely by main projector 12. This processing maycomprise, for example extracting highlight components from image data 42to provide base image 44. In some embodiments the processing comprisesestimating the luminance that will be delivered to image pixels byhighlight projector 20 when driven to display highlight image 43 andcompensating for that estimated luminance in generating base image 44.In some embodiments the estimation may model properties of the opticalsystem of highlight projector 20. In some embodiments the estimation mayestimate light delivered by highlight projector 20 to pixels outside ofhighlight areas. Colour and luminance of main projector 12 and highlightprojector 20 may be calibrated to facilitate such smooth transitions.

Highlight image 43 may take a variety of forms. In some embodiments,highlight image 43 may comprise or be treated as a binary image (allpixels that are “ON” being set to the same level). Such embodiments maybe used, for example, in combination with a process for selectinghighlight areas that selects highlight areas as being made up of pixelsthat have luminances well above the adaptation point. Such embodimentsmay exploit the fact that the human visual system responds similarly tolight that is well above the adaptation point. For example, a viewer maynot be able to tell much or any difference between an image in whichcertain highlight pixels have luminance of 10000 nits and another imagein which the same highlight pixels have luminance of 15000 nits as longas the highlight pixels have luminances well above the adaptation pointin both images. Some such embodiments may operate by distributingluminance from a highlight projector equally over highlight pixelsand/or by clipping the luminance of the highlight pixels to a set level.

In other embodiments the highlight projector may be controlled to supplydifferent luminances to different highlight pixels or areas. In yetother embodiments the highlight projector may be controlled according toa combination of approaches. For example, the highlight processor may becontrolled to supply different luminances to highlight pixels for whichthe image data specifies luminance in a first range and to supply thesame luminance to highlight pixels for which the image data specifiesluminance above the top of the first range. The first range may be fixedor may vary. For example, a variable first range may be based on acurrent adaptation point, on a number of pixels identified as being inhighlight areas, on statistics of pixels identified as being inhighlight areas (e.g. maximum, mean, average, or the like of thehighlight pixels) on combinations of these and the like.

Image data processing may be distributed in various manners. Forexample, in some embodiments, an image processing system 40 isintegrated with a highlight projector such that image data 42 isprovided directly to the highlight projector which derives highlightimage 43 internally. In some alternative embodiments, processing isperformed upstream such that highlight image data 43 is suppliedtogether with base image data 44. For example, highlight image data 43may be encoded together with base image data 44 in a stream, file orother data structure. In such embodiments a projector system may beconfigured to extract the highlight image data 43 and to control ahighlight projector using the base image data 43 while causing a mainprojector to display images according to base image data 44.

A highlight projector may take many different forms. Some examples ofdifferent technologies that may be used for a highlight projectorinclude: scanning spot projectors (some example embodiments of suchprojectors are described above); holographic projectors (e.g. projectorswhich phase modulate light in the Fourier plane of an optical system andthereby concentrate light to form images on an image surface).

An alternative type of scanning projector comprises a 1D light modulatorthat produces a stripe of spatially-modulated light on screen 18 and ascanner that scans the stripe across screen 18. By way of non-limitingexample, the 1D modulator may comprise a 1D polarization modulator incombination with a polarizing beam splitter and a scanning mirror.

Another example embodiment is illustrated in FIG. 5 . FIG. 5 showsschematically a highlight projector 50 comprising a light-redirectingprojector. One general type of such projectors includes projectors whichconcentrate light in some areas to the exclusion of other areas byapplying a diffraction-based/phase modulation approach. This approach issometimes referred to as “holographic 2D projection”.

In the embodiment illustrated in FIG. 5 , the highlight projectorcomprises a coherent light source 51 (in the illustrated embodiment,light source 51 comprises a laser 51A and a beam expander 51B), a phasemodulating panel 52 located in an optical Fourier plane in a light pathof the projector, and a controller 54 which spatially varies thephase-shifting effect of phase modulator 52 according to the realcomponent of the inverse Fourier transform of the desired highlightimage. Controller 54 may be configured to determine a Fourier-basedhologram (sometimes called a computer-generated hologram) correspondingto the highlight image and to set the phase at different locations onphase modulating panel 52 according to the computer generated hologram.The interaction of light from light source 51 with phase modulatingpanel 52 controlled according to the Fourier-based hologram produced bycontroller 54 results in a recreation of the highlight image. Lens 22projects the resulting image onto screen 18 (not shown in FIG. 5 ).

In some embodiments a highlight projector comprises one or moreholographic projectors having variable-intensity light sources. Theintensity of the light source(s) may be controlled to provide furthercontrol over the display of the highlight image.

In some embodiments, a highlight projector comprises a plurality ofholographic projectors which each project light of a different color.For example, one holographic projector may comprise a red light source51 and be controlled to display a red channel of a highlight image. Sucha projector may be used in combination with holographic projectors whichcomprise green and blue light sources and are respectively controlled toimage green and blue channels of the highlight image.

Current projectors of the type which generate images by varying a phasemodulator have the disadvantage that there can be significant leakage oflight due to the finite resolution of the phase modulator and/or becausethe inverse Fourier transform of a desired image will, in general, haveboth real and imaginary parts and a typical phase modulator onlyimplements one part of the inverse Fourier transform. Such light leakagemay be partially or substantially entirely compensated for in highlightprojectors in embodiments in which phase modulated light is imaged toilluminate a spatial light modulator (such as a DMD array, LCOSmodulator, LCD panel or the like). In such embodiments the spatialmodulator may be operated to clean up the projected highlight image byreducing the amount of light outside of highlight areas. The spatialmodulator used for this purpose may be the same as or different from thespatial modulator used in a main projector.

Light leakage may be reduced by providing a phase modulator panel 52having a high spatial resolution. In some embodiments, phase modulatorpanel 52 has a spatial resolution exceeding that of the highlight image.In some embodiments, the number of controllable elements of phasemodulator panel 52 is 9 times or more larger than the number of pixelsin the highlight image.

A holographic projector may optionally be configured to project ahighlight image onto a non-planar focal surface. Controller 54 may beconfigured to generate drive signals for the phase modulator whichresult in focusing onto a desired non-planar surface. For example, theholograph may be configured to yield a focused image on a curved screenor spatial light modulator.

In the embodiment illustrated in FIG. 6A, a holographic projector 72projects a desired highlight image onto a spatial light modulator 74which is controlled to either redirect or dump or absorb light in imageareas outside of the highlight areas. Light from spatial light modulator74 is then imaged onto a screen 18 for example by a projection lens 22.Spatial light modulator may, for example, be controlled by performing asimulation of the operation of holographic projector 72 to obtain anestimate of the actual distribution of light produced by holographicprojector 72. This estimate may then be compared to the highlight image.The comparison may comprise, for example, determining a ratio ordifference of the estimate and the highlight image. Spatial lightmodulator 74 may be controlled according to a result of the comparisonto compensate for differences between the light pattern actuallyprojected by holographic projector 72 and the desired highlight image.Computing the estimate may, for example be performed using a programmeddata processor, hard-configured logic circuits and/or configurable logiccircuits (e.g. field-programmable gate arrays (FPGA's). The computationmay comprise estimating a phase-shifted light field produced by a phasemodulator of holographic projector 72 and computing a Fourier transformof the estimated light field.

In some embodiments, the spreading of light outside of highlight areasis reduced by blocking a DC component in the Fourier plane.

In the example embodiment shown in FIG. 6B, holographic projector 72projects a highlight image directly onto a spatial light modulator 76 ofa main projector. Spatial light modulator 76 is also illuminated bylight source 73.

FIG. 7 shows schematically a projector 60 having an alternativearchitecture. Projector 60 comprises a light source 62 (which does notneed to be a coherent light source). Light source 62 illuminates a 2Dspatial light modulator 64 such as an analog DMD mirror array. Spatiallight modulator 64 has controllable elements that can guide light todifferent positions on screen 18. In some embodiments, spatial lightmodulator 64 is directly imaged onto screen 18 by projection lens 66 toprovide a highlight image. In some embodiments, spatial light modulator64 illuminates another spatial light modulator 65. Spatial lightmodulator 65 may comprise a spatial light modulator also used by a mainprojector, for example (not shown in FIG. 7 ).

A highlight projector 80 according to a further alternative embodimentis illustrated in FIG. 8 . Highlight projector 80 comprises a lightsource 82 that illuminates a spatial light modulator 83. In an exampleapplication, spatial light modulator 83 is controlled so that all pixelsoutside of highlight areas are set to not pass light to screen 18. Sincespatial light modulator 83 is not perfect, some light is passed bypixels outside of the highlight areas. This leakage light may beperceived by viewers as a raising of the black level (e.g. blacks takeon a grey appearance over the entire image). Highlight projector 80includes a spatial filter 84 which, in the illustrated embodimentincludes a mask 85 provided in a Fourier plane in the optical pathbetween spatial light modulator 83 and screen 18. Mask 85 blocks the DCspatial frequency component (i.e. a component of the signal affectingall pixels in the displayed image) thereby decreasing the black levelwhile still passing the highlights.

Systems in which a light source for a main projector or holographicprojector comprises a source of coherent light may include one or moreoptical components configured to reduce the appearance of laser specklein the projected images. Any suitable speckle-reduction technology maybe applied. For example, a variety of techniques for reducing laserspeckle are known in the art. These include techniques such asproviding: a vibrating diffuser in an optical path; randomizing a phaseof the coherent light source, and randomizing polarization of thecoherent light source.

Highlight projectors as described herein may be applied to 3D projectionsystems as well as to 2D projection systems. In embodiments whereviewers wear polarizing or spectrally sensitive glasses such thatdifferent components of the projected light are directed to the viewers'left and right eyes, a highlight projector may be controllable to emitlight for viewing by viewers' left eyes, right eyes, or both eyes. Inthe alternative, separate highlight projectors may be provided toproject highlight images for users' left and right eyes. In someembodiments the highlight projector(s) emit light having differentspectral compositions for viewing by viewers' left and right eyes. Forexample, projection systems as described herein may be used inconjunction with 3D image projection systems as described, for example,in WO2008/140787; WO2011/002757; and U.S. Pat. No. 7,784,938; all ofwhich are hereby incorporated herein by reference for all purposes.

FIG. 9 shows a display 100 according to another embodiment. Display 100may be, for example, a television, computer display, advertising displayor the like. Display 100 may be used with or without a highlightprojector. Display 100 comprises a spatial light modulator panel 102illuminated by a backlight assembly 104. Spatial light modulator panel102 may comprise a transmission-type light modulating panel such as aLCD panel, for example. Backlight assembly 104 comprises a holographicprojector as described herein, for example. The holographic projectorcomprises a coherent light source 106, and a phase-modulating panel 108.Light from light source 106 is phase modulated by panel 108 and directedonto spatial light modulator panel 102.

A display controller 109 receives an image to be displayed, determines adesired backlight light distribution, and controls the holographicprojector to project the desired backlight light distribution onto thespatial light modulator panel 102. The desired backlight lightdistribution may be slowly varying (i.e. comprised mainly of lowerspatial frequencies). A mask 107 (which may be fixed or controllable)may optionally be provided in the Fourier plane to attenuate oreliminate Fourier components corresponding to higher spatialfrequencies. Controller 109 may, for example, determine the desiredbacklight light distribution by low-pass spatial filtering the imagedata, applying a blur filter to the image data, and/or computing localaverages or weighted averages of local groups of pixels in the imagedata or the like. Driving values for pixels of phase-modulating panel108 may be determined by computing an inverse Fourier transform of thedesired backlight light distribution.

In some embodiments, the controller computes an estimate of the actuallight distribution at spatial light modulator panel 102. This estimatemay be used in setting pixels of spatial light modulator panel 102 toprovide an image according to the image data. For example, the value fora pixel of spatial light modulator panel 102 may be set by comparing theintensity of light estimated to be incident on the pixel from backlight104 to the intensity of light that the image data specifies for thepixel and setting the pixel of the spatial light modulator panel toreduce the intensity of the incident light to the intensity specified bythe image data. The comparison may comprise, for example, dividing theimage data by the estimated incident light intensity.

Computing the estimated incident light intensity may comprise estimatinghow light-modulating panel 108 will affect light from light source 106when driven by the driving signals established by the controller andusing that information to compute a light field resulting from theapplication of the signals to phase-modulating panel 108. The lightfield at spatial light modulator 102 may then be estimated by computinga Fourier transform of the light field.

In some embodiments, display 100 comprises a color display. In some suchembodiments spatial light modulator panel 102 comprises a monochromespatial light modulator. In such embodiments backlight 104 may comprisethree or more monochrome light sources (e.g. red, green and blue lasers)that can each be operated to illuminate phase-modulating panel 108. Animage may be displayed by time multiplexing images of different colors.For example, a red image may be displayed based on a red channel ofimage data using a red light source 106. This may be followed insuccession by a green image displayed based on a green channel of imagedata using a green light source 106 and a blue image displayed based ona blue channel of image data using a blue light source 106. Thecontroller may take into account the wavelength of the light from eachlight source 106 in setting pixels of phase-modulating plate 108 tophase-modulate light from the light source. In some embodimentsbacklight 104 comprises a separate unit (e.g. a holographic projector)for each of a plurality of primary colors.

It is not mandatory that highlight image data for driving the highlightprojector be derived from image data in real time during display of animage. The highlight image data may be determined in advance andprovided as part of the image data, or separately provided. Inembodiments which employ a holographic highlight projector, image valuesfor controlling a phase-modulating panel may be determined in advanceand provided as part of the image data.

Certain implementations of the invention comprise computer processorswhich execute software instructions which cause the processors toperform a method of the invention. For example, one or more processorsin a display system may implement image processing methods as describedherein executing software instructions (which may be or include firmwareinstructions) in a program memory accessible to the processors. Theinvention may also be provided in the form of a program product. Theprogram product may comprise any medium which carries a set ofnon-transitory computer-readable signals comprising instructions which,when executed by a data processor, cause the data processor to execute amethod of the invention. Program products according to the invention maybe in any of a wide variety of forms. The program product may comprise,for example, physical media such as magnetic data storage mediaincluding floppy diskettes, hard disk drives, optical data storage mediaincluding CD ROMs, DVDs, electronic data storage media including ROMs,PROMs, EPROMs, flash RAM, or the like. The computer-readable signals onthe program product may optionally be compressed or encrypted.

Where a component (e.g. a software module, processor, assembly, device,circuit, etc.) is referred to above, unless otherwise indicated,reference to that component (including a reference to a “means”) shouldbe interpreted as including as equivalents of that component anycomponent which performs the function of the described component (i.e.,that is functionally equivalent), including components which are notstructurally equivalent to the disclosed structure which performs thefunction in the illustrated exemplary embodiments of the invention.

While a number of exemplary aspects and embodiments have been discussedabove, those of skill in the art will recognize certain modifications,permutations, additions and sub-combinations thereof. It is thereforeintended that the following appended claims and claims hereafterintroduced are interpreted to include all such modifications,permutations, additions and sub-combinations as are within their truespirit and scope.

The invention claimed is:
 1. A method for displaying images, comprising:spatially modulating first light according to a desired image with afirst spatial light modulator; Fourier transforming the first lightrepresenting the desired image onto a Fourier plane of a Fouriertransform component; producing second light by attenuating oreliminating one or more frequency components of the Fourier transformedlight by a mask positioned on the Fourier plane; spatially modulatingthe second light according to the desired image with a second spatiallight modulator; generating the first light with a first light source;and generating third light with a second light source; and wherein thesecond spatial light modulator is configured to spatially modulate boththe second light and the third light according to the desired image, thesecond light and the third light being spatially overlapped on thesecond spatial light modulator.
 2. The method of claim 1, wherein thefirst light is Fourier transformed onto the Fourier plane based ondiffraction.
 3. The method of claim 1, wherein the mask is configured toattenuate or eliminate Fourier components corresponding to one or morespatial frequencies.
 4. The method of claim 1, wherein the mask is afixed or controllable optical mask.
 5. The method of claim 1, whereinthe first light is Fourier transformed onto the Fourier plane by a phasemodulating panel.
 6. The method of claim 5, wherein driving values forpixels of the phase modulating panel are determined by an inverseFourier transform of a desired light distribution.
 7. The method ofclaim 5, wherein phase-shifting effect of the phase modulating panel isspatially varied according to a real component of an inverse Fouriertransform of the desired image.
 8. The method of claim 1, furthercomprising: controlling, with a controller, the first spatial lightmodulator to perform phase modulation based on the desired image; andcontrolling, with the controller, the second spatial light modulator toperform intensity modulation based on the desired image.
 9. A displaysystem, comprising: a first spatial light modulator configured tospatially modulate first light according to a desired image; a Fouriertransform component configured to Fourier transform the first lightrepresenting the desired image onto a Fourier plane; a mask positionedon the Fourier plane and configured to produce second light byattenuating or eliminating one or more frequency components of theFourier transformed light; a second spatial light modulator configuredto spatially modulate the second light according to the desired image; afirst light source configured to generate the first light and a secondlight source configured to generate third light and wherein the secondspatial light modulator is configured to spatially modulate both thesecond light and the third light according to the desired image, thesecond light and the third light being spatially overlapped on thesecond spatial light modulator.
 10. The display system of claim 9,wherein the first light is Fourier transformed onto the Fourier planebased on diffraction.
 11. The display system of claim 9, wherein themask is configured to attenuate or eliminate Fourier componentscorresponding to one or more spatial frequencies.
 12. The display systemof claim 9, wherein the mask is a fixed or controllable optical mask.13. The display system of claim 9, wherein the first light sourceincludes a laser and a beam expander.
 14. The display system of claim 9,further comprising: a reflecting or transmitting screen opticallycoupled to the second spatial light modulator for displaying the desiredimage.
 15. The display system of claim 9, wherein the Fourier transformcomponent to Fourier transform the first light onto the Fourier planeincludes a phase modulating panel.
 16. The display system of claim 15,wherein driving values for pixels of the phase modulating panel aredetermined by an inverse Fourier transform of a desired lightdistribution.
 17. The display system of claim 15, wherein phase-shiftingeffect of the phase modulating panel is spatially varied according to areal component of an inverse Fourier transform of the desired image. 18.The display system of claim 9, further comprising a controllerconfigured to: control the first spatial light modulator to performphase modulation based on the desired image; and control the secondspatial light modulator to perform intensity modulation based on thedesired image.